Although this invention is being disclosed in connection with cervical cancer, it is applicable to many other areas of medicine. Visual inspection is common practice in the field of medicine to detect and/or diagnose a disease. For example, a colposcopic examination involves a systematic visual evaluation of the lower genital tract (cervix, vulva, and vagina) with the purpose of identifying and ranking the severity of lesions, so that biopsies representing the highest-grade abnormality can be taken, if necessary.
At times, visualization enhancement techniques are employed in medical imaging to improve the identification of diagnostically significant tissue structures. In colposcopy, different colored filters are used to accentuate blood vessel patterns that cannot be easily seen by using regular white light. A green filter is commonly used and blocks all light except green light and allows for the increased visualization of cervical blood vessel patterns. Three different abnormal blood vessel patterns can be distinguished during a colposcopic exam: mosaic, punctation and atypical; all of which are major diagnostic features in the identification of pre-cancerous regions of the cervix (Reid, R., Herschman, B. R., Crum, C. P., Fu, Y. S., Braun, L., Shah, K. V. et al., Genital warts and cervical cancer. V. The tissue basis of colposcopic change, Am. J. Obstet. Gynecol. 149(3): 293-303. 1984, incorporated herein by reference; Reid, R. and Scalzi, P., Genital warts and cervical cancer. VII. An improved colposcopic index for differentiating benign papillomaviral infections from high-grade cervical intraepithelial neoplasia, Am. J. Obstet. Gynecol. 153(6): 611-618. 1985, incorporated herein by reference; D. G. Ferris, J. T. Cox, D. M. O'Connor, V. C. Wright, and J. Foerster, Modern Colposcopy. Textbook and Atlas, pp. 1-699, American Society for Colposcopy and Cervical Pathology, 2004, incorporated herein by reference).
The implementation of the green filter visual enhancement is commonly accomplished by filtering the standard white light by placing a green-colored optical filter at the output end of the light source, i.e. illuminating the cervix with green light, or by placing a green-colored filter between the cervix and the colposcope detection optics, i.e. filtering the reflected white light before being detected, either visually by the operator or electronically with an imaging sensor. These approaches have a number of drawbacks: First, the use of colored filters reduces the amount of light available and could result in a decrease in image quality. Second, colored filters add cost and mechanical complexity in that a filter assembly must be incorporated which allows for both standard white light and green filtered light during the colposcopic exam. Third, colored filters are limited to the colors of the filters.
In digital colposcopy, in which images or videos of the cervix are electronically captured using a light sensitive imaging sensor and displayed on an external monitor, other visualization enhancement techniques can be employed. The current invention applies image algorithms in the form of mathematical morphology operations in order to enhance the contrast between the cervical blood vessel patterns and the surrounding tissue, and as such provides a non-physical implementation for the increased visualization of blood vessel structures, similar to the effect of using an optical green filter.
Mathematical morphology is a technique for the analysis and processing of geometrical structures and has been used, as related to the current invention, in the automated analysis of vessels from angiograms (K. Sun and N. Sang, Enhancement of vascular angiogram by multiscale morphology, in Bioinformatics and Biomedical Engineering, 1311-1313 (2007), incorporated herein by reference), micro-calcification analysis from mammograms (Wirth, M., Fraschini, M., and Lyon, J., Contrast enhancement of microcalcifications in mammograms using morphological enhancement and non-flat structuring elements, Proc. 17th IEEE Symposium on Computer-Based Medical Systems, (2008), incorporated herein by reference) and brain magnetic resonance imaging (J. D. Mendiola-Santibanez, I. R. Terol-Villalobos, G. Herrera-Ruiz, and A. Fernandez-Bouzas, Morphological contrast measure and contrast enhancement: One application to the segmentation of brain MRI, Signal Processing 87, (2008), incorporated herein by reference).
By implementing the visualization enhancement as an algorithm applied to an image of the cervix offers several advantages compared to the optical green filter approach. First, the algorithm approach will not reduce the amount of light and will maintain the image quality of the captured images or videos. Second, no incremental cost or mechanical complexity is added. Third, the algorithm implementation can provide the capabilities of variable visualization enhancements similar to different filter characteristics (such as color) by changing one or several algorithm parameters. Fourth, the algorithm approach can be applied to the standard white light image, providing an image display that is more pleasingly to the human eye. Fifth, image pre-processing steps can be performed to further improve the visualization enhancement. One such step would be to segment the images into different regions and apply the visualization enhancement to some of these regions only; for example, only those regions with a high likelihood of finding pre-cancerous or cancerous lesions.
Applying a visual enhancement algorithm as described in the current invention to uterine cervical images can provide the physician with increased visualization of blood vessel patterns that are sometimes difficult to detect by the naked eye. Such an algorithm could assist the physician in the identification of diagnostically important structures and provide important information in the diagnostic process.
Although many of the algorithms described in the present invention are well-known in the art, the inventors are unaware of another contrast enhancement method combining a pre-processing step to identify regions with high likelihood of disease, a mathematical morphology step to identify high and low intensity regions, and applying a tuning parameter for variable visualization enhancements, all as described later, which achieves unexpectedly better results.
The following patents and patent applications may be considered relevant to the field of the invention:
U.S. Pat. No. 6,147,705 to Krauter et al, incorporated herein by reference, discloses a video colposcope which includes a system microcomputer having algorithms for color balance levels stored into memory. A video camera obtains a subject electronic image of a subject object, and using algorithm-driven digital signal processing circuitry (DSP), color saturation, hue, and intensity levels of the subject electronic image are modified according to previously stored DSP reference filter algorithms and reference color balance levels, thus producing a modified electronic image corresponding to the subject electronic image. The modified electronic image is outputted to a display in continuous real time as the corresponding subject image is obtained by the video camera. This modified electronic image emulates that obtained through an optical green filter and incorporates a simulated white balance.
U.S. Pat. No. 6,277,067 to Blair, incorporated herein by reference, discloses a method and portable apparatus for the visual examination and grading of cervical epithelium by means of a hand-held colposcope assembly capable of producing a digital image of the cervix. This invention enables real-time imaging and archiving of the entire cervix for the purpose of detecting cancerous and pre-cancerous tissue and by virtue of computerized image processing to suggest an objective diagnosis of the cervical epithelium by means of a low cost, portable, hand-held digital colposcope.
U.S. Pat. No. 6,032,070 to Flock et al., incorporated herein by reference, discloses a system and method to view an anatomical structure such as blood vessels in high contrast with its surrounding tissue. The system and method is used to produce an image of an anatomical structure using reflected electromagnetic radiation singularly scattered from target tissue. The system and method also provide same-side illumination and detection of reflected electromagnetic radiation in a convenient integral imaging device. The system and method also provides helmet mounted imaging technology in a single integral helmet which allows the wearer to view an anatomical structure located within a patient such that the image is continuously orientated according to the orientation to the helmet wearer's head. The system and method is also used in the performance of venipuncture. The system and method provide improved contrast between any anatomical structure and its surrounding tissue for use in any imaging system.
U.S. Pat. No. 7,305,111 to Arimura et al., incorporated herein by reference, discloses a method, system, and computer program product for detecting at least one nodule in a medical image of a subject, including identifying, in the medical image, an anatomical region corresponding to at least a portion of an organ of interest; filtering the medical image to obtain a difference image; detecting, in the differenced image, a first plurality of nodule candidates within the anatomical region, calculating respective nodule feature values of the first plurality of nodule candidates from the first plurality based on pixel values of at least one of the medical images and the differenced image; removing false positive nodule candidates from the first plurality of nodule candidates based on the respective nodule feature values to obtain a second plurality of nodule candidates; and determining the at least one nodule by classifying each of the second plurality of nodule candidates as a nodule or a non-nodule based on at least one of the pixel values and the respective nodule feature value. True-positive nodules are identified using linear discriminant analysis and/or a MultiMTANN.
US Patent Application Ser. No. 2005/0100208 to Susuki et al., incorporated herein by reference, discloses a method, system, and computer program product for modifying an appearance of an anatomical structure in a medical image, e.g., rib suppression in a chest radiograph. The method includes: acquiring, using a first imaging modality, a first medical image that includes the anatomical structure; applying the first medical image to a trained image processing device to obtain a second medical image, corresponding to the first medical image in which the appearance of the anatomical structure is modified; and outputting the second medical image. Further, the image processing device is trained using plural teacher images obtained from a second imaging modality that is different from the first imaging modality. In one embodiment, the method also includes processing the first medical image to obtain plural processed images, wherein each of the plural processed images has a corresponding image resolution; applying the plural processed images to respective multi-training artificial neural networks (MTANNs) to obtain plural output images, wherein each MTANN is trained to detect the anatomical structure at one of the corresponding image resolutions; and combining the plural output images to obtain a second medical image in which the appearance of the anatomical structure is enhanced.
US Patent Application Ser. No. 2006/0018548 to Chen et al., incorporated herein by reference, discloses a method, system, and computer software product for analyzing medical images, including obtaining image data representative of a plurality of medical images of the abnormality, each medical image corresponding to an image of the abnormality acquired at a different time relative to a time of administration of a contrast medium, each medical image including a predetermined number of voxels; partitioning each medical image into at least two groups based on the obtained image data, wherein each group corresponds to a subset of the predetermined number of voxels, and each group is associated with a temporal image pattern in the plurality of medical images; selecting, from among the temporal patterns, an enhancement temporal pattern as representative of the abnormality; and determining, based on the selected temporal pattern, a medical state of the abnormality.
US Patent Application Ser. No. 2006/0147101 to Zhang et al., incorporated herein by reference, discloses a method for computer-aided detection of microcalcification clusters that obtains digital mammography data for a single view image and normalizes and filters the image data to reduce noise. A first mask is generated and applied to the image data for defining the breast structure, forming a first cropped image. A second mask is generated and applied to the image data for defining muscle structure, forming a second cropped image. An artifact mask corresponding to vascular calcifications and known imaging artifacts is generated and applied to the first and second cropped images, defining first and second artifact-masked cropped images. In a repeated sequence, portions of each artifact-masked cropped image are processed using an enhancement algorithm and reducing edge effects to obtain a set of microcalcification cluster candidates and suspected microcalcification clusters. Image processing algorithms remove false positives from the listing of microcalcification clusters and classify candidate microcalcification clusters to identify true positives.
US Patent Application Ser. No. 2006/0184002 to Yarnykh et al., incorporated herein by reference, discloses a reduced field-of-view (FOV) imaging technique that combines suppression of signals from outer volume and inflowing blood. Both outer volume and blood suppression are achieved using an SFQIR (Small-FOV Quadruple-inversion-Recovery) preparative pulse sequence including two double-inversion pulse pairs separated by appropriate delays. Within each pair, inversion pulses are successively applied to the imaged slice and the slab orthogonal to the imaging plane, with the thickness equal to the FOV size in the phase-encoding direction. Each double-inversion results in a reinversion of the magnetization in a central part of the FOV, while outer areas of the FOV and inflowing blood remain inverted. The SFQIR module was implemented for single-slice and multislice acquisition with a fast spin-echo readout sequence. Timing parameters of the sequence corresponding to the maximal suppression efficiency can be found by minimizing variation of the normalized signal over the entire range of T.sub.1 occurring in tissues.
US Patent Application Ser. No. 2007/0165921 to Agam et al., incorporated herein by reference, discloses a method for improving a thoracic diagnostic image for the detection of nodules. Non-lung regions are removed from the diagnostic image to provide a lung image. Vessels and vessel junctions of the lung(s) in the lung image are enhanced according to a first-order partial derivative of each of a plurality of voxels of the lung image. A vessel tree representation is constructed from the enhanced vessels and vessel junctions. The vessel tree representation can be subtracted from the lung image to enhance the visibility of nodules in the lung(s).
US Patent Application Ser. No. 2008/0051660 to Kakadaris et al., incorporated herein by reference, discloses a set of intravascular ultrasound (IVUS) related systems, apparatuses and methods. New catheter designs including contrast agent introduction subsystems and/or Doppler subsystems are disclosed. Methods for acquiring and analyzing Doppler data from intravascular ultrasound (IVUS) catheters are disclosed. RF-based detection of blood and/or contrast agents such as micro-bubbles are disclosed. Methods for frame-grating image data analysis permitting frame registration before, during and after a contrasting effect is imposed on a system being imaged are disclosed. Methods for difference imaging for contrast detection are disclosed. Methods for quantification and visualization of IVUS data are disclosed. And methods for IVUS imaging are disclosed.
US Patent Application Number to Shen et al., incorporated herein by reference, discloses a method and system for automatically detecting rib metastasis in a thoracic CT volume. The ribs are segmented in said CT volume by recursive tracing. A series of cross-sectional images are then generated along a centerline of each rib. Cortical and trabecular bone structures are segmented in each of the cross-sectional images for each rib. Features are calculated for each cross-sectional image based on characteristics of the cortical and trabecular bone structures, and alterations are detected in the cross-sectional images based on the features. Rib metastasis is detected in a rib when an alteration is detected in a number of consecutive cross-sectional images along the centerline of the rib.